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Abstract 


The  goal  of  this  MLIRI  project  is  to  develop  the  tools  required  for  the  systematic  design  of 
cooperative  control  systems  for  distributed  vehicles  in  adversarial  environments.  The  project 
is  a  collaborative  effort  between  the  participating  institutions  of  Caltech,  Cornell,  MIT,  and 
UCLA.  The  following  “dimensions”  of  cooperative  control  offer  an  effective  breakdown  of 
the  cooperative  control  landsacpe:  1)  Distributed  control  and  computation,  2)  Adversarial 
interactions,  3)  Uncertain  evolution,  and  4)  Complexity  management.  The  Eve  year  MLIRI 
effort  has  identified  and  addressed  many  subproblems  in  the  cooperative  control  landscape 
and  has  produces  both  simulation  and  hardware  testbeds. 
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Selected  Research  Highlights 


Over  the  past  5  year,  the  MURI  has  involved  26  students,  8  postdocs,  and  13  university 
faculty  members,  resulting  in  over  100  publications. 

The  following  are  selected  highlights  of  research  conducted  under  this  MURI: 


Consensus  in  Switching  Topology  Networks  Reynolds’s  Flocking  Rules:  Vehi¬ 
cle  flocking  can  be  viewed  in  the  context  of  a  network  of  communicating  agents  in  which  com¬ 
munication  links  persistently  change.  A  basic  paradigm  for  decision  making  over  networks 
is  the  notion  of  “consensus”,  in  which  the  collective  of  agents  reaches  a  mutual  agreement 
despite  the  switching  network  topology.  This  work  introduced  the  notion  of  a  “balanced” 
graph  and  established  that  consensus  is  achievable  under  this  property.  This  work  also  led  to 
the  first  complete  analytical  justification  of  the  widely  popular  “Flocking  Rules  of  Reynolds” 
(separation,  alignment,  &  cohesion),  introduced  in  1987  as  a  heuristic  for  inducing  flocking 
behavior  among  virtual  vehicles. 

Representative  publications: 

•  R.  Olfati-Saber  and  R.M.  Murray,  “Consensus  problems  in  networks  of  agents  with 
switching  topology  and  time-delays” ,  IEEE  Transaction  on  Automatic  Coritrol,  Septem¬ 
ber  2004. 

•  R.  Olfati-Saber,  “Flocking  for  multi-agent  dynamic  systems:  Algorithms  and  theory” , 
IEEE  Transactions  on  Automatic  Control ,  March  2006. 


Computation  and  control  language  programming  environment:  Design  of  dis¬ 
tributed  systems,  indeed  any  design  process,  will  involve  design  iterations  and  exploratory 
simulations.  This  work  facilitates  such  a  process  tailored  to  distributed  systems  through  a 
programming  environment  based  on  the  Computation  and  Control  Language  (CCL).  CCL  is 
a  guarded-command  language  for  expressing  systems  wherein  control  and  computation  are 
intertwined.  The  environment  exploits  automated  verification  methods  to  aid  in  the  devel¬ 
opment  of  distributed  decision  making  protocols  with  guaranteed  performance  properties. 

Representative  publications: 
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•  E.  Klavins,  “A  language  for  modeling  and  programming  cooperative  control  systems”, 
Proceedings  of  the  International  Conference  on  Robotics  and  Automation ,  2004. 

•  E.  Klavins  and  R.M.  Murray,  “Distributed  algorithms  for  cooperative  control”,  IEEE 
Pervasive  Computing ,  January  2004. 

•  E.  Klavins,  “A  formal  model  of  a  multi-robot  control  and  communication  task”,  Pro¬ 
ceedings  of  the  42nd  IEEE  Conference  on  Decision  and  Control ,  December  2003. 


Mixed-integer  LP  methods  for  multivehicle  systems:  This  work  considers  trajectory 
planning  for  multiple  vehicles  in  the  presence  of  an  invading  team.  The  objective  is  to  disable 
invaders  before  they  reach  a  critical  defense  zone.  This  work  introduces  an  abstraction  of 
this  problem  called  the  “RoboFlag  Drill”.  The  RoboFlag  drill  serves  as  a  motivating  and 
illustrating  challenge  problem  for  several  components  of  the  MURI  activities.  Because  of  the 
binary  nature  of  an  invader  being  active  or  disabled,  the  resulting  optimization  has  a  com¬ 
binatorial  aspect.  This  work  shows  how  mixed-integer  linear  programming  tools,  which  are 
commercially  available,  can  be  used  to  derive  nominal  intercepting  trajectories.  Complemen¬ 
tary  work  shows  how  simplifying  assumptions  can  lead  to  reduced  complexity  computations, 
using  only  LP’s. 

Representative  publications: 

•  M.  G.  Earl  and  R.  D’Andrea,  “A  Study  in  Cooperative  Control:  The  RoboFlag  Drill”, 
Proceedings  of  the  American  Control  Conference,  2002. 

•  M.  Earl  and  R.  DAndrea,  “Iterative  MILP  methods  for  vehicle  control  problems”, 
IEEE  Transactions  on  Robotics,  2005. 

•  G.  Chasparis  and  J.  Shannna,  “Linear-Programming-Based  Multi- Vehicle  Path  Plan¬ 
ning  with  Adversaries”,  Proceedings  of  the  American  Control  Conference,  June  2005. 


Finite  state  mode  estimation:  Because  of  either  limited  communications  or  restricted 
information,  there  is  uncertainty  between  vehicles  in  knowledge  of  each  other’s  state.  Here, 
the  intent  is  not  traditional  notions  of  state  estimation,  but  rather  “state”  is  being  used  as 
“mode  of  operation” .  Understanding  the  mode  of  operation  is  important  for  determining 
consensus  among  teammates  or  assessing  the  intent  from  adversaries.  This  work  takes  two 
complementary  approaches  to  mode  estimation.  The  first  approach  uses  completely  finite 
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state  models.  While  one  may  construct  a  sort  of  set-valued  observer,  such  an  approach 
would  be  computationally  prohibitive.  Rather,  this  work  constructs  an  observer  that  takes 
advantage  of  the  special  structures  that  allow  simplified  observer  representations.  The  second 
approach  is  a  hybrid  setting  of  continuous  dynamics  with  probabilistic  mode  changes.  This 
work  develops  a  mode  observer  that  capture  the  tradeoff  between  rates  of  mode  transitions 
and  accuracy  of  mode  estimations. 

Representative  publications: 

•  D.  Del  Vecchio,  R.  M.  Murray  and  E.  Klavins,  “Discrete  state  estimators  for  systems 
on  a  lattice”,  Automatic a,  2006. 

•  N.C.  Martins  and  M.  A.  Dahleh,  “An  Information  Theoretic  Approach  to  the  Mode 
Estimation  of  Randomly  Switching  FIR  Systems” ,  Proceedings  of  the  IEEE  Mediter¬ 
ranean  Control  Conference,  2003. 

•  D.  Choukroun,  Speyer,  J.L.,  “Mode- Estimator-Free  Quadratic  Control  of  Jump-Linear 
Systems  with  Mode-Detection  Random  Delay”,  Proceedings  of  the  AIAA  Guidance, 
Navigation,  and  Control ,  2005. 


Distributed  convergence  to  Nash  equilibria:  Noncooperative  game  theory  has  long 
been  a  paradigm  for  investigating  strategic  mechanisms  with  hostile  adversaries.  In  min- 
max  or  zero-sum  games,  there  is  a  tacit  assumption  that  each  adversary  knows  the  intention 
of  the  other.  An  important  question  is  what  happens  when  intentions  are  not  shared. 
One  can  still  define  concepts  such  as  a  Nash  equilibrium  in  this  case,  but  an  unresolved 
question  in  game  theory  is  how  a  Nash  equilibrium  could  emerge  through  interactions  of 
bounded  rationality  players.  This  work  has  shown  that  Nash  equilibria  can  emerge  by 
viewing  adversarial  interactions  as  a  feedback  system.  The  resulting  analysis  and  simulations 
produce  convergent  behaviors  in  long  standing  (40  years)  counterexamples  suggesting  that 
Nash  equilibria  will  not  emerge. 

Representative  publications: 

•  J.S.  Shamma  and  G.  Arslan,  “Unified  convergence  proofs  of  continuous  time  fictitious 
play”,  IEEE  Transactions  on  Automatic  Control,  July  2004. 

•  J.S.  Shamma  and  G.  Arslan,  “Dynamic  fictitious  play,  dynamic  gradient  play,  and 
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distributed  convergence  to  Nash  equilibria”,  IEEE  Transactions  on  Automatic  Control, 
March  2005. 


Differential  games  with  probabilistic  models:  Many  representations  of  adversarial 
behavior  presume  a  “worst  case”  situation.  This  framework  alone  does  not  capture  many 
attrition  models  that  use  probabilistic  models  to  represent  adversarial  encounters.  A  more 
appropriate  optimization  in  this  case  is  a  combination  of  worst  case  modeling  and  expected 
value  analysis.  This  work  considers  such  a  framework  for  linear-quadratic  games,  in  which 
strategies  are  derived  that  optimize  against  both  worst  case  opponent  and  a  neutral,  but 
probabilistic,  environment.  The  work  shows  that — contrary  to  conjectures  in  earlier  litera¬ 
ture  that  suggested  infinite  dimensional  optimal  controllers — the  ensuing  optimal  controllers 
may  well  have  a  familiar  model  based  structure. 

Representative  publications: 

•  A.  Swarup  and  J.L.  Speyer,  “Characterization  of  Linear  Quadratic-Gaussian  Differen¬ 
tial  Games  with  Different  Information  Patterns”,  IEEE  Conference  on  Decision  and 
Control,  2004. 

•  A.  Swarup  and  J.L.  Speyer,  “Handling  Partial  and  Corrupted  Information,”  in  Ad¬ 
vanced  Technology  Concepts  for  Command  and  Control:  An  Anthology  of  Feedback- 
Based  Approach  to  Command  and  Control,  A.  Kott  (editor), CRC /Taylor  and  Francis, 
2005. 


Optimal  navigation  in  partially  unknown  environments:  In  distributed  vehicle  sys¬ 
tems,  each  vehicle  has  local  information  which  can  be  shared  with  team  members.  There 
is  an  inherent  trade-off  in  such  operations,  namely  the  tension  between  dispersion  for  the 
sake  of  information  gathering  and  cohesion  for  the  sake  of  information  exploitation.  In  other 
words,  if  vehicles  form  a  sparse  spread,  then  they  may  gather  more  information  but  are  not 
sufficiently  close  to  benefit  from  shared  information.  This  work  addresses  this  trade-off  in  a 
quantifiable  manner  through  the  investigation  of  navigation  in  partially  unknown  environ¬ 
ments.  The  work  uses  a  dynamic  programming  framework  to  determine  optimal  distancing 
that  balances  the  dispersion/cohesion  trade-off. 

Representative  publications: 
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•  J.  De  Mot  and  E.  Feron,  “Spatial  Distribution  of  Two-Agent  Clusters  for  Efficient 
Navigation”,  Proceedings  of  the  IEEE  Conference  on  Decision  and  Control ,  2003. 

•  J.  De  Mot  and  E.  Feron,  “Optimal  Two-Agent  Navigation  with  Local  Environment 
Information”,  Allerton  Conference  on  Control  and  Communications ,  2004. 


Control  under  communication  constraints:  Coordinated  interaction  of  multiple  ve¬ 
hicles  will  inevitably  require  networked  communications.  This  work  investigates  control 
performance  limitations  imposed  by  controller  implementations  over  a  network.  In  partic¬ 
ular,  this  work  explores  the  trade-off  between  communications  channel  capacity  and  robust 
stabilization  through  analytical  bounds  that  are  not  restricted  to  any  specific  form  of  a 
controller  or  coding  scheme,  and,  as  such,  provide  ultimate  limits  of  achievable  performance. 

Representative  publications: 

•  N.C.  Martins,  M.A.  Dahleh,  and  N.  Elia,  “Stabilization  of  Uncertain  Systems  in  the 
Presence  of  a  Stochastic  Digital  Link”,  Proceedings  of  the  IEEE  Conference  on  De¬ 
cision  and  Control ,  2004.  (Extended  version  submitted  to  the  IEEE  Transactions  in 
Automatic  Control). 

•  N.C.  Martins  and  M.A.  Dahleh,  “Fundamental  Limitations  of  Disturbance  Attenuation 
in  the  Presence  of  Finite  Capacity  Feedback”,  Proceedings  of  the  American  Control 
Conference ,  2005.  (Extended  version  submitted  to  the  IEEE  Transactions  in  Auto¬ 
matic  Control). 

•  S.  Sarnia  and  M.A.  Dahleh,  “Remote  Control  over  Noisy  Communication  Channels: 
A  first  order  example”,  Accepted  for  publication,  IEEE  Transactions  on  Automatic 
Control,  2006. 


Language  Evolution:  This  work  is  indirectly  related  to  the  problem  of  consensus.  The 
question  is  to  understand  the  limiting  behavior  of  interacting  populations  that  use  different 
grammars/languages,  e.g.,  whether  there  will  be  a  single  dominant  language  or  a  distribution 
of  languages.  A  relation  to  multivehicle  systems  is  understanding  how  highly  autonomous 
vehicles  on  extended  missions  will  be  able  to  communicate  the  presence  of  entities  that  were 
not  previously  encountered. 

Representative  publications: 
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•  Y.  Lee,  T..  C.  Collier,  G.  M.  Kobele,  E.  P.  Stabler  and  C.  E.  Taylor.  “Grammar 
structure  in  language  evolution” ,  European  Conference  on  Artificial  Life ,  2005. 

•  Y.  Lee,  T.  C.  Collier,  E.  S.  Stabler  and  C.  E.  Taylor.  “The  role  of  population  structure 
in  language  evolution”,  in  M.  Sngisaka  and  H.  Tanaka,  eds.,  Proceedings  of  the  Tenth 
International  Symposium  on  Artificial  Life  and  Robotics ,  2005. 


Probability  maps  of  environments  with  moving  opponents:  Trajectory  planning  in 

uncertain  environments  requires  the  existence  of  some  sort  of  map  of  threat  probabilities.  In 
the  case  of  a  non-stationary  environment,  such  a  probability  map  must  be  updated  to  reflect 
threat  evolution  due  to  moving  adversaries.  This  work  develops  tools  for  the  construction 
and  evolution  of  such  probability  maps.  The  work  goes  on  to  investigate  the  role  of  such  maps 
in  multivehicle  path  planning.  Again,  a  trade-off  emerges  between  the  potential  benefits  and 
associated  risks  of  exploration.  The  presence  of  moving  opponents  further  complicates  such 
issues. 

Representative  publications: 

•  M.  Jun  and  R.  D’Andrea,  “Probability  map  building  of  uncertain  dynamic  environ¬ 
ments  with  indistinguishable  obstacles” ,  Proceedings  of  the  American  Control  Confer¬ 
ence,  2003. 

•  V.G.  Rao  and  R.  D’Andrea,  “Managing  Multiple  Moving  Vehicles  with  Patch  Repre¬ 
sentations”,  Fifth  Annual  Conference  on  Cooperative  Control  and  Optimization  (CCO 
’05),  2005. 
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Personnel  Supported  During  Duration  of  Grant 


These  include: 


Faculty 

Jason  Hickey,  Caltech  (CS) 

Richard  Murray,  Caltech  (ME/CDS) 
Raffaello  D’Andrea,  Cornell  (MAE) 
Carla  Gomes,  Cornell  (CS) 

Munther  Dahleh,  MIT  (EECS) 

Eric  Feron,  MIT  (AA) 

Steve  Massaquoi,  MIT  (EECS/HST) 
Jeff  Shamma,  UCLA  (MAE) 

Jason  Speyer,  UCLA  (MAE) 

Edward  Stabler,  UCLA  (Lin) 

Charles  Taylor,  LICLA  (Bio) 

Past  participants 
Brian  Williams,  MIT  (AA) 

Greg  Pottie,  UCLA  (EE) 


Postdocs 

Eric  Klavins,  Caltech 
Reza  Olfati-Saber,  Caltech  &  UCLA 
Myungsoo  Jun,  Cornell 
Venkatesh  G.  Ran,  Cornell 
Vishwesh  Kukharni,  MIT 
Nuno  Martins,  MIT 
Gurdal  Arslan,  UCLA 
Jonathan  Wolfe,  UCLA 


Students 


Zhipu  Jin,  Caltech 
Steve  Waydo,  Caltech 
Atif  Chaudhry,  Cornell 
Matthew  Earl,  Cornell 
Jeff  Fowler,  Cornell 
Cedric  Langbort,  Cornell 
Jan  De  Mot,  MIT 
Sommer  Gentry,  MIT 
Georgious  Kotsalis,  MIT 
Nuno  Martins,  MIT 
Zhi-Hong  Mao,  MIT 
Eugene  Lim,  MIT 
Sridevi  Sarma,  MIT 


Holly  Waisane,  MIT 
Ibrahim  Al-Shyoukh,  UCLA 
George  Chasparis,  LICLA 
Travis  Collier,  LICLA 
Gregory  Kobele,  UCLA 
Amish  Pandya,  LICLA 
Yoosook  Lee,  UCLA 
Ying  Lin,  UCLA 
Jason  Marden,  UCLA 
Jason  Riggle,  UCLA 
Vincent  Seah,  UCLA 
Ashitosh  Swamp,  UCLA 
Yuan  Yao,  UCLA 


AA:  Aeronautics  &  Astronautics,  Bio:  Ecology  and  Evolutionary  Biology,  CDS:  Con¬ 
trol  and  Dynamical  Systems,  CS:  Computer  Science,  EECS:  Electrical  Engineering 
and  Computer  Science,  HST:  Health  Sciences  and  Technology,  Lin:  Linguistics,  MAE: 
Mechanical  and  Aerospace  Engineering,  ME:  Mechanical  Engineering. 


Several  of  the  MURI  alumni  have  gone  on  to  faculty  positions  of  their  own.  These  include: 


•  Gurdal  Arslan:  University  of  Hawaii 

•  Daniel  Choukroun:  Ben  Gurion  University 

•  Domitilla  Del  Vecchio:  University  of  Michigan 

•  Eric  Klavins:  University  of  Washington 

•  Nnno  Martins:  University  of  Maryland 

•  Zhi-Hong  Mao:  University  of  Pittsburgh 

•  Reza  Olfati-Saber:  Dartmouth  College 


Publications 


The  MURI  has  resulted  in  over  100  publications.  A  comprehensive  list  and  many  electronic 
copies  are  available  at:  http://www.seas.ucla.edu/coopcontrol/. 

In  addition  to  these  publications,  there  will  be  an  edited  collection  stemming  from  the  work  of 
this  MURI  entitled  Cooperative  Control  of  Distributed  Multi- Agent  Systems  to  be  published 
by  John  Wiley  &  Sons. 


Honors  &  Awards  Received 

•  Arslan:  NSF  CAREER  Award 

•  Dahleh:  IEEE  Fellow,  IEEE  Axelby  Award,  Plenary  Speaker:  MTNS,  SyDyC  Student 
Symposium  (UCLA) 

•  D Andrea:  RoboCup  2nd,  1st,  2nd,  3rd,  1st  (2000-2005),  NSF  CAREER  &  PECASE 
Award,  Plenary  Speaker:  Cellular  Automata  Conf,  Robot  Motion  Control  Conf,  SIAM 
Conf  on  Control  and  its  Applications;  Distinguished  Lecturer  Rome  Labs,  NASA  Lan¬ 
gley,  MTNS. 

•  Klavins:  NSF  CAREER  Award 

•  Martins  &  Dahleh:  Hugo  Schuck  Award 
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•  Murray:  IEEE  Fellow,  Plenary  Speaker:  2003  ECC,  2005  ACC,  2006  ASCC,  SyDyC 
Sutdent  Symposium  (UCLA) 

•  Shamma:  IEEE  Fellow 

•  Speyer:  NAE  Member,  AIAA  Fellow,  IEEE  Fellow 


Interactions  and  Transitions 

•  Arslan:  Lockheed  Martin  Orincon,  “Autonomous  Sensor  Allocation  for  Submarine 
Tracking” . 

•  Dahlch  &  Shamma:  AFOSR  Award  ($600K),  “Control  Theoretic  Modeling  for  Uncer¬ 
tain  Cultural  Attitudes  and  Unknown  Adversarial  Intent”. 

•  Feron:  Nascent  Tech  Corp  Navy  STTR  &  DARPA  SBIR;  Advisory  Board  of  French 
National  Academy  of  Technologies. 

•  Murray  &  Klavins:  AFOSR  MURI,  “High  Confidence  Design  for  Distributed  Embed¬ 
ded  Systems”. 

•  Murray:  Member,  National  Research  Council  (NRC)  Panel  on  Network  Science,  2005; 
Member,  Air  Force  Scientific  Advisory  Board,  2002-present;  Chair,  Panel  on  Future 
Directions  in  Control,  Dynamics  and  Systems,  2000-2002;  Chair,  Collaborative  Center 
on  Control  Science  (CCCS)  Executive  Board,  2002-present. 

•  Shamma:  REEF  Research  Advisory  Board;  Speaker,  NRC  Workshop  on  NASA  Rela¬ 
tionships  for  Technology  Development. 

•  Speyer:  Member,  NRC  Panel  on  Decadal  Survey  of  Civil  Aeronautics;  SySense,  Inc. 
Phase  II  SBIR  with  BMD. 

•  Taylor:  NSF  Award  ($1M),  “Sensor  Arrays  for  Monitoring  Bird  Behavior  and  Diver¬ 
sity”  . 
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